EEVi – framework for evaluating the effectiveness of visualization in cyber-security

Cyber-security visualization is an up-and-coming area which aims to reduce security analysts’ workload by presenting information as visual analytics rather than a string of text and characters. But the adoption of the resultant visualizations has not increased. The literature indicates a research gap of a lack of guidelines and standardized evaluation techniques for effective visualization in cyber-security, as a reason for it. Therefore, this research addresses the research gap by developing a framework called EEVi for effective cyber-security visualizations for the performed task. The term ‘effective visualization’ can be defined as the features of visualization that are crucial to perform a certain task successfully. EEVi has been developed by analyzing qualitative data that leads to the formation of cognitive relationships (called links) between data that act as guidelines for effective cyber-security visualization in terms of the performed task. The methodology to develop this framework can be applied to other fields to understand cognitive relationships between data. Additionally, the analysis presents a glimpse into the usage of EEVi in cyber-security visualization

EEVi: A model developed to aid the design and evaluation process of cyber-security visualisation for cyber-security analysts

The area of visualisation in cyber-security is advancing quickly. At present, there are no standardised guidelines for designing and evaluating visualisations. There is limited end-user involvement in the design process, which leads to visualisations that are generic and often ineffective for cyber-security analysts. This contributes to low adoption of the resulting cyber-security visualisation solutions, highlighting a major research need. This research presents EEVi (Effective Execution of Visualisation), a model developed to aid in the design and evaluation of cyber-security visualisations for cyber-security analysts. ‘Thematic Analysis’, a qualitative data analysis technique, was used to develop EEVi. 13 experts were interviewed (seven cyber-security analysts and six visualisation designers) to validate this model. Their feedback guided revisions to the model and was subsequently used to perform statistical analyses. This demonstrated that there were no statistically significant differences between visualisation designers and cyber-security analysts. Neither was there statistically significant agreement. The individual responses led to modification of the component tasks of the model. The modified model was confirmed by 30 respondents, primarily from cyber-security, through an online questionnaire. This confirmed the model’s relevance, and validity, guiding the revision of the component tasks. The confirmed model, were used to create a work-domain analysis (abstraction hierarchy) diagram and mockups to demonstrate a possible real-world utilisation of EEVi. These were evaluated by 10 experts (five cyber-security analysts and five visualisation designers) and their feedback validated the notion that, with a common structure the disparity of understanding between cyber-security analysts and visualisation designers can be minimised. The questionnaire responses were also used to formulate a quantitative value calculator called C-EEVi (Calculator for EEVi) using the ‘Analytical Hierarchy Process’. C-EEVi can be used to score cyber-security visualisation solutions for a performed task.This work has developed a model, EEVi, to help design cyber-security visualisations for cyber-security analysts to perform a specific task. The abstraction hierarchy diagram of EEVi provides a basis for communication between cyber-security analysts and visualisation designers. Lastly, C-EEVi evaluates cyber-security visualisation solutions for a task, by allocating them a quantitative value score. These address the major research gaps identified in this thesis.<br/

Expert-interviews led analysis of EEVi - a model for effective visualization in cyber-security

The area of visualization in cyber-security is advancing at a fast pace. However, there is a lack of standardized guidelines for designing and evaluating the resulting visualizations. Furthermore, limited end-user involvement in the design process leads to visualizations that are generic and often ineffective for cyber-security analysts. Thus, the adoption of the resultant cyber-security visualizations is low and this highlights a major research gap. This paper presents expert-interview based validation of EEVi - a model developed to aid in the design and evaluation process of cyber-security visualizations, with a view to make them more effective for cyber-security analysts. A visualization is considered effective if the characteristics of the visualization are essential for an analyst to competently perform a certain task. Thirteen experts were interviewed (six visualization designers and seven cyber-security analysts) and their feedback guided revisions to the model. The responses were subsequently transposed from qualitative data to quantitive data in order to perform statistical analyses on the overall data. This demonstrated that the perspectives of visualization designers and cyber-security analysts generally agreed in their views of effective characteristics for cyber- security visualization, however there was no statistically significant correlation in their responses

EEVi �Framework and Guidelines to Evaluate the Effectiveness of Cyber-Security Visualization

Cyber-security visualization aims to reduce security analysts� workload by presenting information as visual analytics instead of a string of text and characters. However, the adoption of the resultant visualizations by security analysts, is not widespread. The literature indicates a lack of guidelines and standardized evaluation techniques for effective visualization in cyber-security, as a reason for the low adoption rate. Consequently, this article addresses the research gap by introducing a framework called EEVi for effective cyber-security visualizations for the performed task. The term �effective visualization� is defined as the features of visualization that are critical for an analyst to competently perform a certain task. EEVi has been developed by analyzing qualitative data which led to the formation of cognitive relationships (called links) between data. These relationships acted as guidelines for effective cyber-security visualization to perform tasks. The methodology to develop this framework can be applied to other fields to understand cognitive relationships between data. Additionally, the analysis of the framework presented, demonstrates how EEVi can be put into practice using the guidelines for effective cyber- security visualization. The guidelines can be used to guide visualization developers to create effective visualizations for security analysts based on their requirements